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Abstract.  Error  bounds  are  developed  for  a  class  of  quadratic 
programming  problems.  The  absolute  error  between  an  approximate 
feasible  solution,  generated  via  a  dual  formulation,  and  the  true 
optimal  solution  is  measured.  Furthermore,  these  error  bounds 
involve  considerably  less  work  computationally  than  existing 
estimates . 
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1.  Introduction.  Quadratic  programming  has  long  been  the  corner¬ 
stone  for  many  numerical  techniques  in  nonlinear  programming.  Typically, 
•>  these  methods  use  a  quadratic  function  to  approximate  the  objective  func¬ 

tion  and  linear  equalities  and  (or)  inequalities  to  approximate  the 

% 

constraint  functions,  all  in  some  neighborhood  of  a  specified  feasible 
point.  The  resulting  quadratic  programming  problem  is  solved  and  the 
optimal  solution  (or  approximation  thereof)  becomes  the  point  from  which 
a  new  round  of  approximations  is  made  and  a  new  quadratic  programming 
problem  is  formulated.  In  addition,  quadratic  programming  merits 
attention  in  its  own  right  as  it  arises  naturally  in  such  fields  as  engi¬ 
neering,  economics,  and  game  theory. 

Increasingly,  though,  the  topic  has  been  treated  as  a  special  case 
of  the  linearly  constrained  variational  inequality  problem: 

Find  u*  such  that 

(u-u*)^  f(u*)  5  0  for  all  u  in  K  (1) 

where  K  is  a  polyhedral  set 

K  =  {u:  Cu  ^  d,  Au  =  b)  (2) 

and  C,  A,  d,  and  b  are  specified  matrices  and  vectors  of  appropriate 
dimensions.  Of  special  interest  has  been  the  development  of  error 
bounds  for  approximate  solutions,  x,  to  the  true  optimal  solution, 

X*,  of  (1).  Particular  attention  has  been  paid  to  the  case  where  f(x) 
is  a  linear  function 

» 

f(x)  =  Mx+c 

with  M  a  positive  definite  (p.d.)  matrix.  Recently,  Pang  [8l  provided 
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a  survey  of  known  error  estimates  along  with  a  number  of  extensions. 

These  follow  the  general  format 

II  x-x*  II  ^  p-r(x) 

where  p  is  a  constant  independent  of  x  (but  dependent  on  the  data)  and 
r(x)  is  the  generalized  residual,  a  quantity  depending  on  the  value  of 
X  and  the  type  of  measure  employed.  A  common  drawback  to  these  bounds 
is  that  at  least  one  of  the  two  quantities  is  difficult  to  calculate 
(and  any  simplification  in  one  seems  to  make  the  other  more  difficult). 

The  purpose  of  this  paper  is  to  derive  a  set  of  error  estimates 
for  a  special  case  of  the  quadratic  programming  problem: 

min  ^  x^Hx  +  v^x  (3) 

St.  Ax  =  b 
X  §  0 

where  H  is  positive  definite,  and  A  €  is  of  full  row  rank.  We  will 

sometimes  refer  to  this  as  the  primal  (quadratic  programming)  problem. 

In  relationship  to  Pang's  work  we  remark  that  the  results  presented 
here  are  a  hybrid  of  those  termed  linear  programming  measures  (of  error) 
and  dual  measures  (of  error)  in  [8].  A  Corollary  to  our  Theorem  3.3 
results  in  one  similar  to  Pang's  theorem  4.1.  However,  our  bounds  are 
relatively  easy  to  compute  and  thus  can  be  employed  not  only  for  a  poster 
iori  error  analysis  but  also  as  a  test  for  termination  of  the  particular 
quadratic  programming  algorithm  being  used. 

We  have  divided  the  remainder  of  the  paper  into  two  sections.  In 
section  2  we  develop  a  dual  formulation  to  (3)  using  conjugate  func¬ 
tions.  This  approach  allows  us  to  estimate  the  current  duality  gap 
(denoted  by  A)  and  consequently  bounds  the  improvement  we  can  expect  in 
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our  dual  objective  functional.  As  the  dual  formulation  is  solved  it 
generates  approximate  optimal  solutions  to  the  primal  problem  (given  by 
the  gradient  of  the  dual  objective  functional).  In  section  3  we  bound 
the  error  between  these  generated  solutions  (denoted  by  x)  and  the  true 
optimal  solution  to  the  primal  problem  (denoted  by  x")  as  a  function 
of  the  estimated  duality  gap  (A). 

2.  The  Dual  Formulation.  In  this  section  we  derive  the  dual  for¬ 
mulation  to  system  (3).  The  theory  of  duality  in  quadratic  programming 
is  not  particularly  new.  Dorn  [2]  and  Lemke  [5]  are  among  the  first  to 
address  this  topic.  Both  formulations  in  these  works  are  different  from 
the  dual  in  this  section,  yet  both  can  be  used  to  derive  it.  Lost  in  the 
derivation,  however,  is  the  notion  of  the  duality  gap  which  is  essential  to 
our  analysis.  Therefore,  we  will  use  the  general  theory  of  duality  as  set 
forth  in  Avriel  [1].  To  employ  this  theory  we  introduce  perturbation  vari¬ 
ables  Wj ,  w^  and  v  and  define 


<l)(x,w^  .w^.v)  = 


r  T  T 

^x  Hx  +  V  X 

+  w 


if  Ax-b^.. 
-Ax+b^w 
x^ 

otherwise . 


The  conjugate  function,  (p*,  becomes 


rsup(C^x+\^(Ax-b)+A.2("Ax+b)-u^-(^x^Hx+v^)  )  for 
X 


+  06 


Otherwise . 


Letting  y  =  ^^”^2  substituting  t,  =  0  (in  anticipation  of  the  Weak 


Duality  Theorem),  we  get 
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r 


T  T  T  T 

sup[y  (Ax-b)+u  x-(*sx  Hx+v  x)  for  u^O 

X 


t)"(0,y,u)  =  < 


+  00 


otherwise . 


The  supremiun  over  x  can  now  be  removed  by  performing  the  necessary 
maximization  and  inserting  the  optimal  solution,  yielding 


(t)*(o,y,u)  =  < 


^y^AH'^Ay-y^b+y^AH“^(u-v)+^y^AH'^A^y 


+  00 


for  u  ^  0 


otherwise . 


By  the  Weak  Duality  Theorem  we  have 


inf  <t>(x, 0,0,0)  ^  sup  -([)''■  (o,y,u). 
X  (y,u) 


Since  y  is  free,  it  too  can  be  eliminated  by  performing  the  required 
maximization  of  -({>*;  one  can  easily  show  that  the  correct  (optimal) 
choice  of  y  (in  terms  of  u  S  0)  is 

y*  =  (AH"\a^)"^  (b-AH'^(u-v)). 


Substituting  this  result  and  acknowledging  the  dependence  of  0*  on  u 
yields 


^u^Pu+c^u  +  k  for  u  ^  0  (AA) 

(t)*  (u)  =  •( 

+  <*>  otherwise 

where  P  =  h‘^  -  h"^A^(AH“^A^)"^AH'^  (4B) 

c  =  -Pv  +  H“^A^(AH'^A^)"^b  (4C) 

k  =  ijv^Pv  -  b^(AH~^A^)'^AHv^  -  ^b^(AH" ^A^)" ^b .  (4D) 
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The  weak  duality  inequality  becomes 

XT  T  T 

inf  [^x  Hx+v  x]  ^  sup  -[^u  Pu  +  c  u  +  k] 

Ax=b  u^O 

x^O 

and  by  virtue  of  this  inequality,  any  x  satisfying  the  primal  constraints 
[Ax=b,  x^O]  and  any  u^O  provides  a  "working  duality  gap"  -  or  an  upper  bound 
on  the  possible  improvement  in  either  minimizing  the  left  hand  side  (start¬ 
ing  from  x)  or  maximizing  the  right  hand  side  (starting  from  u) .  As  such, 
we  define 

A(x,u)  =  ^x^Hx  +  v^x  +  iju^Pu  +  c^u  +  k  (5) 

for  each  (x,u)  satisfying  Ax=b,  xiO,  and  u^O.  Moreover,  we  define  the  dual 
quadratic  program  to  (3)  as 

inf  *su^Pu  +  c^u  +  k  (6) 

s.t.  u^O 

Clearly,  A(x,u)  =  0  means  x  and  u  are  optimal  to  the  primal  and  dual 
problems,  respectively. 

The  next  proposition  is  an  elementary  exercise  in  linear  algebra 
whose  proof  is  left  to  the  reader. 

Proposition  2.1.  The  following  relationships  hold 

1.  AP  =  0 

2.  \\m  II  h‘^  II 

3.  PHP  =  P 

4.  A  (Pu+c)  =  b 

T  T 

5.  c  HP  =  -V  P 

T  T 

6.  *5cHc  +  vc  =  -k 
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If  we  assume  that  the  primal  problem  (3)  has  a  feasible  solution,  then 
the  dual  problem  (6)  is  bounded  below  and  it  is  not  hard  to  show  that 
there  exists  an  optimal  solution  u*.  This  u--  also  solves  the  (equivalent)  ^ 

linear  complementarity  problem:  Find  u  such  that 

Pu+c  ^  0  u^O  u^(Pu+c)  =  0. 

By  virtue  of  proposition  2.1  (part  4.),  X”  =  Pu'-'+c  is  a  feasible  point  to 
the  primal  problem  and  a  direct  calculation  shows  A(x“,U“)  =  0  demonstra¬ 
ting  the  optimality  of  x-'^  as  well.  As  a  consequence  of  the  same  proposition 
(part  1),  we  observe  that  the  function  Pu+c  is  not  strongly  monotone  over  the 
nonnegative  orthant,  as  evidenced  by  the  nontrivial  nullspace  of  P. 

For  the  remainder  of  the  paper  we  will  make  the  following  assumption: 

(A)  There  exists  an  interior  point  x  to  (3):  Ax=b,  x>0. 

We  remark  that  assumption  (A)  is  equivalent  to  either  of  the  following 
assumptions 

(A')  The  set  of  optimal  solutions  to  the  dual  problem  (6)  is 

bounded  or 

(A")  There  exists  a  u^O  such  that  Pu+OO. 

The  details  can  be  found  in  [10].  Observe  that  (A")  implies  (A’)  as 
previously  shown  by  Mangasarian  and  McLinden  in  [6]. 

3.  Error  Estimates.  A  number  of  reasons  warrant  computing  solutions 
to  (3)  via  the  dual  formulation  (6).  The  constraint  set  consists  only  of 
nonnegativity  constraints  allowing  for  a  considerable  reduction  in  the  compu¬ 
tational  work  load.  Moreover,  the  dual  problem  preserves  the  number  of  var- 

I 

iables ,  unlike  many  existing  dualities,  and  has  accessible  feasible  points. 
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Recall  from  section  2  that  an  optimal  dual  solution  u-'  induces  the 
corresponding  (unique)  optimal  solution  to  the  primal  via  the  formula 
x*=Pu*+c.  As  such,  we  want  to  know  how  close  the  estimate  x=Pu+c  is  to 
the  primal  optimal  solution.  Observe  that  the  estimate  satisfies  Ax=b  by 
Proposition  2.1  (part  4)  but  not  necessarily  the  nonnegativity  condition, 
x^O.  A  number  of  quadratic  programming  algorithms  applied  to  (6)  force 
the  nonnegativity  of  Pu+c,  particularly  those  based  on  adding  a  logarithmic 
barrier  function  (see  Ko j ima  et  al.  [4]  or  Monteiro  and  Adler  [7]).  Others 
based  on  projecting  (or  truncating  components  of)  the  gradient  do  not.  Of 
the  former  type,  the  two  cited  references  actually  enforce  positivity  which, 
as  remarked  in  section  2,  requires  that  assumption  (A)  hold.  Error  esti¬ 
mates  in  this  case  are  handled  easily  by  Corollary  3.4. 

To  start,  we  will  derive  a  bound  on  ||x-x'''||  (again,  x=Pu+c)  based  on 
the  true  duality  gap,  i.e.,  the  difference 

^u^Pu+c^u  -  [^(u*)^Pu*+c^u-) . 

Lemma  3.1.  Let  u*  denote  an  optimal  solution  to  (6),  namely 
T  T 

rain  iju  Pu+c  u  subject  to  u^O, 

with  P  and  c  defined  as  in  (4b, 4c).  At  u  suppose  an  upper  bound  6  has  been 
acquired  such  that 

^u^Pu+c^u  -  [ !s(u'')^Pu*+c^U''']  ^  6  (7) 

(as  for  example,  when  one  has  an  upper  bound  on  the  duality  gap).  Then 

||P(u*-u)||  S  -J  2  6 

where  \  is  the  largest  eigenvalue  of  the  (p.s.d.)  matrix  P. 
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Proof ■  It  is  easy  to  show  that 

6  S  ^u^Pu+c'^u-[J5(u*)^Pu*+c^u*]=-5i(u-u*)^P(u-U")^-(Li''''-u)^(Pu+c) .  (8) 

\ 

Define  F(t)  on  [0,1]  by 

F(t)=^(u+t[u--uj )^P(u+t[u"-u] )+c^(u+t[u*-u] ) . 

Observe  that  u+t[u"-u]  is  nonnegative  for  te[0,l]  and  F(t)  is  monotone 
decreasing  on  [0,1] .  Thus  F'(t)^0  on  [0,1]  which,  for  t=l,  yields  the 
inequality 

(u*-u)^P(u''''-u)^-(u''-u)^(Pu+c) .  (9) 

The  two  inequalities  (8)  and  (9)  imply 

(u*-u)^P(u*-u)S26.  (10) 

Finally,  since  P  is  p.s.d.  (and  symmetric)  we  have 

(u*-u)^P^P(u*-u)SA.  (u*-u)^P(u*-u) 
m 

from  which  we  conclude 

||P(u*-u)||^g26\  . 

m 

Taking  the  square  root  of  both  sides  of  the  inequality  completes  the  proof. 

Corollary  3.2.  If  x=Pu+c  is  an  approximate  optimal  solution  to  (3) 

(not  necessarily  nonnegative),  then 

||x-x*||SV2A.  6 

m  * 

where  x*  is  the  (unique)  optimal  solution  to  (3)  and  6  satisfies  (7). 
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Our  focus  now  turns  to  obtaining  good  estimates  of  the  6  in  (7). 

The  working  duality  gap  A(x,u)  is  one  such  bound,  but  it  requires  that  x 
be  feasible  to  Ax=b,  x^O.  If  x  is  not  nonnegative  (but  satisfies  Ax=b), 
then  we  can  use  a  perturbation  technique  analogous  to  that  in  [3,  Thm.  29] 
or  [9]  to  create  a  nearby  point  which  is.  More  precisely,  suppose  that  an 
interior  point  x  has  been  generated  (or  as  is  the  case  in  some  problems, 
known  in  advance).  Then  for  x=Pu+c,  the  perturbed  point  z  given  by 

z=tx+( l-t)x 


is  both  feasible  and  nonnegative  provided  O^t^0  where 

r 


0  =  min  ^ 
i 


X  . 
1 


X . -min{x . ,0} 
1  1 


(11) 


Taking  t=0  to  define  a  particular  z,  and  then  using  it  to  evaluate  A(z,u) 
is  the  basis  for  the  following  theorem. 

Theorem  3.3.  Suppose  assumption  (A)  holds  and  that  an  interior 
point  X  has  been  specified.  Let  x  be  defined  by  x=Pu+c  and  let  x*  be 
the  (unique)  optimal  solution  to  the  primal  quadratic  programming  problem 
(3).  Then  there  exists  a  constant  vector  ^  and  constants  p  and  T,  all 
depending  on  x,  such  that 


||x-x*||i  { x'^u+  ( 1  -0)  [ 0x'^C+  ( 1  +0) k+  ( 1  -0) p+T ] } 

where  0  is  given  by  equation  (11),  k  is  the  constant  defined  in  (4d),  and 

\  is  the  largest  eigenvalue  of  P. 
m 

Proof.  In  light  of  Lemma  3.1  it  suffices  to  show  that  the  quantity 
in  brackets  {•}  is  an  upper  bound  similar  to  6  in  (7).  Define  z=©x+(l-0)x 
where  0  is  given  (depending  on  x  and  x)  by  (11).  Then  A(z,u),  as  given 
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in  (5 ) ,  becomes 

A(z  ,u)=^0^u^PHPu+0^c^HPu+*50^c^Hc+0(  1  -0)x'^H(Pu+c)+5j(  1  -0)^x^Hx+0\’^(Pu+c ) 

+  ( l-0)v^x+*su^Pu+c^u+k. 

'J'  'J*  rri  t 

Using  the  identities  PHP=P,  c  HP=-v  P,  and  c=-k-v  c  from  proposition 
2.1  and  regrouping: 

A(z  ,u)  =  (ij0^+5s)u"^Pu+c^u+(  1-0)  (0(Pu+c)^(Hx+v)  +  (  l+0)k+^5(  l-0)x^Hx+x^v]  . 

Finally,  using  the  positive  semi-definiteness  of  P  and  the  definition  of 
X,  we  obtain 

A(z  ,u)Su^x+(  1-0)  [0x^(Hx+v)  +  (l+0)k+(l-0)  (J5X^Hx)+x"*'v]  , 

the  constants  in  brackets  {'}  now  being  evident  if  we  take 

C=Hx+v  p=Jsx^Hx  t=x^v.  (12) 

Corollary  3.4.  Suppose  for  a  given  u^O  Pu+c^O.  Then  for  x=Pu+c 

||x-x'''||^  JlK  x"^u 
m 

Proof:  0=1  if  Pu+c^O,  and  the  term  under  the  root  in  Theorem  3.3 
reduces  accordingly. 

Observe  that  no  attempt  has  been  made  to  fit  the  interior  point  x  to  a 
particular  value  of  Pu+c,  nor  have  we  tried  to  optimize  our  choice  of  0 
for  a  given  x.  For  example,  given  x  and  Pu+c  one  might  minimize 
the  function 

4 

g(s)=(l-s)[s  x^(Hx+v)+(l+s)k+(l-s) (^x^Hx)+x^v] 

subject  to  the  constraint  O^s^0,  possibly  improving  the  error  bound. 

Additionally,  in  response  to  a  candidate  Pu+c,  one  might  attempt  to 
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generate  an  interior  point  with  larger  positive  components  correspond! ig 
to  (and  in  scale  with)  the  negative  components  of  Pu+c,  thus  reducing  the 
factor  1-0.  However,  without  additional  a  priori  knowledge  of  the  parti¬ 
cular  approximate  solutions  (Pu+c)  to  be  encountered,  it  seems  wise  to 
find  a  "good"  interior  point  (i.e.,  one  with  uniformly  large  positive 
entries)  in  an  attempt  to  keep  the  factor  1-0  small.  Such  an  interior 
point  can  be  found  by  solving  (either  exactly  or  approximately)  the 
linear  program 

Max  t 

s.t.  Ax  =  b  (13) 

x-te^O 
x.t^O. 

T 

where  e  is  the  vector  of  I's;  e  =( 1 , 1 , . . . , 1 ) .  Observe  that  any  feasible 
point  (x,t)  to  (13)  with  t>0  yields  a  set  of  constants  via  (12)  which  may 
be  used  to  estimate  the  absolute  error  between  our  current  candidate 
(Pu+c)  and  the  true  ontimal  solution  (x*) • 

Observe  that  Theorem  3. 3  (and  Corollary  3.4)  can  be  extended  to  incor¬ 
porate  the  notion  of  the  residual,  i.e.,  the  portion  of  Pu+c  which  violates 
the  nonnegativity  condition.  To  measure  the  violation  consider  the  norm 

ll(Pu+c)’||p 

where  (Pu+c)  denotes  the  vector  obtained  by  inserting  O's  in  the  positive 
components  of  Pu+c.  In  particular,  if  the  (»-norm  is  chosen,  and  (x,t)  is  a 
feasible  solution  to  (13)  with  t>0  then 


t 


0  ^  min 
i 


X.  +  ||(Pu+c)'||^ 


thus  the  factor  (1-0)  in  Theorem  3.3  becomes 


1-0  ^ 


ll(Pu+c)'|l^ 

5.  +  ||(Pu+c)‘||^ 


ll(Pu+c)'il^ 


Various  extensions  and  alternate  formulations  of  Theorem  3.3  and  Corollary 
3.4  are  now  obvious.  We  will  not  pursue  these  here. 

We  conclude  with  a  remark  about  the  computational  efficiency  of  these 

error  estimates.  As  previously  mentioned,  once  an  interior  point  to 

Ax=b  x^O  has  been  found,  it  may  be  used  to  bound  the  error  between  any 

candidate  (Pu+c)  and  x*.  The  error  bounds  are,  of  course,  more  relevant 

to  those  approximate  solutions  which  are  nearly  nonnegative  (thus  1-0 

T 

will  be  small)  and  nearly  complementary  (hence  x  u  will  be  small).  Each 
of  the  constants  in  Theorem  3.3  can  be  computed  (or  bounded)  with  a 
minimal  amount  of  work. 

In  contrast,  error  estimates  which  rely  on  the  gap  function  (see 
section  4  of  [8])  require  the  complete  solution  to  a  linear  programming 
problem  for  each  approximate  optimal  solution  considered.  The  dual 
measures  (see  section  5  of  [8])  again  require  a  substantial  amount  of 
work.  Here,  one  needs  a  constant  defined  by  the  maximum  norm  of  a  vector 
constrained  to  the  surface  of  an  ellipsoid  (characterized  by  the  data). 

The  potential  of  the  error  bounds  in  Theorem  3.3  can  be  realized  with 
minimal  computational  work.  They  should  be  considered,  as  our  title  sug¬ 


gests,  practical. 
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